Computational study of psychosis symptoms and facial expressions

Nonverbal behavior is an essential part of face-to-face clinical interactions, especially in mental health settings. Facial expressions can reveal information about a person’s emotions, mental state and social intentions, and is routinely used by both patients and their doctors in a variety of ways that can impact healthcare outcomes. In our work we demonstrate how automated tools of facial expression analysis can help in assessing a number of clinical scales that measure severity of psychosis symptoms. We analyze nonverbal behaviors during semi-structured clinical interviews and find interesting results at the whole session level and when looking at individual responses to clinicians questions. We demonstrate the importance of such analysis and highlight a number of behavioral indicators related to various psychosis symptoms.